Image processing system for detecting stationary state of moving object from image, image processing method, and recording medium

ABSTRACT

In order to detect retention in a preferable manner, an image processing system is provided with: a retention area extraction unit that determines whether an area is a retention area in an image frame of a processing time on the basis of a first image generated from each of image frames taken within a first time width from the processing time and a second image generated from each of image frames taken within a second time width from the processing time which is longer than the first time width; and a reliability calculation unit that generates reliability information relating to the determination of the retention area for each area in the image frames to be processed.

TECHNICAL FIELD

Some example embodiments according to the present invention relate to animage processing system, an image processing method and a recordingmedium.

BACKGROUND ART

In video surveillance, for example, identifying a left-behind object anda person who stays in a place for a period of time longer than a certaintime has been considered in recent years (see PTL 1, for example). InPTL 1, motion in a scene is analyzed at multiple time scales andlong-term and short-term background models are compared with oneanother. This is used to differentiate among pixels that belong toprimarily static background portions of the scene, pixels that belong tothe active foreground portions, and pixels that belong to a left-behindobject which has been static for some amount of time.

PTL 2 to PTL 6 also discloses related art.

CITATION LIST Patent Literature

PTL 1: Japanese Patent Publication No. 5058010

PTL 2: International Publication No. WO 2009/125569

PTL 3: Japanese Laid-open Patent Publication No. 2013-065151

PTL 4: Japanese Laid-open Patent Publication No. 2012-212238

PTL 5: Japanese Laid-open Patent Publication No. 2010-015469

PTL 6: Japanese Laid-open Patent Publication No. 2009-230759

SUMMARY OF INVENTION Technical Problem

In the approach described in PTL 1, when a flow of dense moving objects(for example a flow of people) occurs, the flow of the moving objectsblends into the background. A flow of moving objects blends into thebackground differently between a long-term background model and ashort-term background model, which results in noises. It is thereforedifficult to properly detect a left-behind object by comparing thelong-term background model and the short-term background model.

Some aspects of the present invention have been made in light of theproblems described above and one of the objects of the present inventionis to provide an image processing system, an image processing method anda recording medium for properly detecting a stationary state.

Solution to Problem

An image processing system according to one aspect of the presentinvention includes: determining means for determining whether or not aregion is a stationary region in an image frame at the time ofprocessing, based on a first image generated from each image framecaptured in a first time period from the time of the processing and asecond image generated from each image frame captured in a second timeperiod from the time of the processing, the second time period beinglonger than the first time period; and generation means for generatingreliability information about determination as to whether or not eachregion in an image frame being processed is the stationary region.

An image processing method according to one aspect of the presentinvention includes: determining whether or not a region is a stationaryregion in an image frame at the time of processing, based on a firstimage generated from each image frame captured in a first time periodfrom the time of the processing and a second image generated from eachimage frame captured in a second time period from the time of theprocessing, the second time period being longer than the first timeperiod; and generating reliability information about determination as towhether or not each region in an image being processed is the stationaryregion.

A computer-readable non-transitory recording medium according to thepresent invention stores a program which causes a computer to executethe processes of: determining whether or not a region is a stationaryregion in an image frame at the time of processing, based on a firstimage generated from each image frame captured in a first time periodfrom the time of the processing and a second image generated from eachimage frame captured in a second time period from the time of theprocessing, the second time period being longer than the first timeperiod; and generating reliability information about determination as towhether or not each region in an image frame being processed is thestationary region.

The terms “unit”, “means”, “device” and “system” as used in the presentinvention mean not only physical means but also encompasses softwareimplementations of functions of the “unit”, “means”, “device” and“system”. Further, functions of a single “unit”, “means” “device” and“system” may be implemented by more than one physical means or device;or functions of more than one “unit”, “means”, “device” and “system” maybe implemented by a single physical means or device.

Advantageous Effects of Invention

According to the present invention, an image processing system, an imageprocessing method and a program for properly detecting a stationarystate can be provided.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a functional block diagram schematically illustrating aconfiguration of an image processing system according to a first exampleembodiment.

FIG. 2 is a diagram illustrating a specific example of a method forgenerating a background image.

FIG. 3 is a functional block diagram schematically illustrating aspecific example of a configuration of a static region extraction unit.

FIG. 4 is a flowchart illustrating a flow of processing in the imageprocessing system illustrated in FIG. 1.

FIG. 5 is a flowchart illustrating a flow of processing in the imageprocessing system illustrated in FIG. 1.

FIG. 6 is a flowchart illustrating a flow of processing in the imageprocessing system illustrated in FIG. 1.

FIG. 7 is a block diagram illustrating a hardware configuration capableof implementing the image processing system illustrated in FIG. 1.

FIG. 8 is a functional block diagram schematically illustrating aconfiguration of an image processing system according to a secondexample embodiment.

EXAMPLE EMBODIMENT

Example embodiments of the present invention will be described below.Identical or similar components are respectively given identical orsimilar reference numerals in the following description and referenceddrawings.

(1 First Example Embodiment)

FIGS. 1 to 7 are diagrams for explaining a first example embodiment. Thepresent example embodiment will be described in the following order withreference to the drawings. First, section “1.1” gives an overview of astationary state determination method according to the first exampleembodiment. Then, section “1.2” gives an overview of a functionalconfiguration of an image processing system according to the presentexample embodiment and section “1.3” describes a flow of processing.Section “1.4” describes a specific example of a hardware configurationcapable of implementing the image processing system. Lastly, section“1.5” and subsequent sections describe advantageous effects of thepresent example embodiment and other subjects.

(1.1 Overview)

An image processing system according to the present example embodimentis intended to detect a stationary state of a moving object from imagescaptured by a surveillance camera or the like, for example. The movingobjects the image processing system detects a stationary state thereofmay include, for example, persons, animals, vehicles or the like whichmove on their own, objects left behind by persons, and the like. Thedescription here will mainly focus on detection of a stationary state ofa person.

A conceivable approach to detecting a stationary object may be asfollows, for example. First, many images captured in a long term(hereinafter also referred to as a long-term time window) are averagedor otherwise to generate a background image. The background image iscompared with a background image obtained by averaging the respectiveimages captured in a shorter term (hereinafter also referred to as ashort-term time window) to extract an object staying in images for along time within the short term. By generating average images(background images) from images captured in a certain period of time inthis way, influence of moving objects (including persons) and the likethat quickly move out of frame can be minimized in the images, forexample and therefore a stationary object can be readily extracted. Notethat an image in which the influence of moving objects is minimized bybeing generated from a plurality of images will be referred to as abackground image in the present example embodiment.

However, in cases where movement of moving objects, for example a flowof people constantly occurs, the approach leaves influence of a flow ofpeople significantly in an image generated by averaging captured images.Accordingly, the flow of people blends into the background differentlybetween a long-term time window image and a short-term time windowimage, and as a result, failure of detection of a static object andfalse detection tend to occur.

To address this, the image processing system according to the presentexample embodiment calculates the degree of reliability ofstationary-state detection for each region in an image. By using amethod of excluding portions with low degrees of reliability fromdetermination of a stationary sate and the like, false detection of astationary state can be minimized.

(1.2. System Overview)

A system configuration of the image processing system 100 according tothe present example embodiment will be described below with reference toFIG. 1. FIG. 1 is a block diagram illustrating a system configuration ofthe image processing system 100.

The image processing system 100 includes an image input unit 110, aforeground/background separation unit 120, a stationary regionextraction unit 130, a reliability calculation unit 140, an imagestorage unit 150, a stationary-state likelihood determination unit 160,and an output unit 170.

The image input unit 110 receives an input of time-series frame imagescontained in video input from an imaging apparatus such as asurveillance camera, not depicted. The respective frame images areimages captured at different times. Alternatively, the image input unit110 may receive an input of frame images obtained by decoding video datastored on a hard disk drive (HDD) or a video cassette recorder (VCR),not depicted.

The foreground/background separation unit 120 sequentially separateseach of input images input from the image input unit 110 into aforeground region and background region by using a background differencemethod, an optical flow, or the like, for example. The foreground regionis a dynamic region in an image and the background region is amotionless (static) region (hereinafter also referred to as a staticregion). Determination as to whether or not motion is present can bemade based on whether or not the amount of motion exceeds a threshold,for example. The threshold used for the determination as to whether ornot motion is present may be changed depending on the type of a movingobject in a stationary state to be detected. Specifically, if a rigidobject such as a left-behind suitcase is to be detected, a region inwhich there is no motion in images may be determined to be a backgroundregion and the other regions may be determined to be foreground regions.On the other hand, if a non-rigid object such as a person or a bag is tobe detected, a region in which a change in appearance occur in an imagedue to a slight shaking, a change in attitude or a change in shape butthe amount of change is smaller than a predetermined threshold isdetermined to be substantially statistic and therefore to be abackground region, and the other regions may be determined to beforeground regions. The other regions are regions that have amounts ofchange greater than the threshold. In other words, the term backgroundregion refers to not only a completely motionless region but also aregion that has an amount of motion smaller than a threshold.

In an approach that uses optical flow to separate a dynamic, foregroundregion and a static, background region, an image frame being processedis compared with a previous image frame in a macroblock unit to identifya dynamic block, for example. More specifically, a dynamic block can beidentified by checking where a macroblock (a set of pixels) similar toeach macroblock in an image frame being processed is located in theprevious image frame (block matching), for example.

If background difference method is used, a foreground region and abackground region can be separated by determining whether or not adifference between a background image acquired previously and an imagebeing processed exceeds a threshold by comparison on a pixel-by-pixelbasis or on a pixel-set-by-pixel-set basis, such as macroblock bymacroblock.

The stationary region extraction unit 130 generates a plurality ofbackground images from image frames included in a plurality of timewindows and compares the background images with one another to determinea stationary region.

A specific example of a method for determining a stationary regionperformed by the stationary region extraction unit 130 will be describedbelow with reference to FIG. 2. As illustrated in FIG. 2, for example,the stationary region extraction unit 130 generates a background imagein each of a plurality of time windows from each image frame from whichonly a background region is extracted. The background images can begenerated, for example, by obtaining the average, median, mode value orthe like of pixel values for each pixel in the image frames, in a staticregion in each of the image frames captured in a certain past period oftime from the time of processing. By generating a background image for ashort-term time window and a background image for a long-term timewindow, the image processing system can compare the background images toextract pixels with differences greater than a threshold, thereby canidentify a pixel region made up of the pixels as a stationary region inwhich a moving object is stationary.

While only two background images, i.e. a background image in ashort-term time window and a background image in a long-term timewindow, are depicted in the example in FIG. 2, background images inthree or more time windows may be generated depending on the stationarytime of a moving object to be detected that is stationary.

FIG. 3 illustrates a specific example of a functional configuration ofthe stationary region extraction unit 130. In the example in FIG. 3, thestationary region extraction unit 130 includes a background imagegeneration unit 131, a background image comparison unit 133, and astationary region determination unit 135.

The background image generation unit 131 generates a background image byusing a background region (an image of a static region) extracted by theforeground/background separation unit 120 and background regionsassociated with image frames captured in a plurality of predeterminedtime windows and stored in the image storage unit 150. In this case, thebackground image generation unit 131 can generate a background image bycalculating the average, median or mode value of pixel values in thetime windows obtained at respective pixel positions associated with thebackground region in each of the image frames.

The image storage unit 150 stores images associated with backgroundregions in respective input images extracted one by one by theforeground/background separation unit 120 in a predetermined period oftime.

The background image comparison unit 133 compares background imagesgenerated in time windows by the background image generation unit 131with one another. More specifically, the background image comparisonunit 133 compares a background image generated from the longest timewindow with a background image generated from a shorter time window todetect a stationary region in which a moving object is static for agiven period of time. The background image generated from the longesttime window is a background image assumed to be composed of a truebackground that does not include a moving object. The background imagecomparison unit 133 may detect a stationary region by generatingbackground images from backgrounds in a plurality of time windows andclassifying the background images according to the duration of a staticstate.

The background image comparison unit 133 may compare background images,for example, by a method that uses the absolute value of a difference inpixel value between the background images or a method that calculates acorrelation between pixel values in a rectangular region of small sizewhile scanning the rectangular regions on an image. Alternatively, thebackground image comparison unit 133 may also compare background imagesby using a method that calculates the histogram distance between pixelvalues in a rectangular region, for example. For the method using arectangular region, a predetermined size such as a macroblock may beset. For a method that uses a rectangular region, different sizes may beset for different locations in an image frame by using a cameraparameter and taking into consideration what size a moving object to bedetected appears in the image frame. One way to set different sizes indifferent locations in an image frame may be to set a larger rectangularregion in an image region in which the foreside of a scene appears andset a smaller rectangular region in an image area in which the rear sideof the scene appears, for example.

The stationary region determination unit 135 identifies pixels withdifferences greater than a threshold as a result of comparison by thebackground image comparison unit 133 and determines a pixel region madeup of such pixels as a stationary region, for example. In this case, iffeatures such as the size and shape of an object to be detected areknown, the stationary region determination unit 135 may make thedetermination excluding a pixel region that does not match the features.

Referring back to FIG. 1, the reliability calculation unit 140calculates the degree of reliability of stationary-state detection ineach region (including the case of being each pixel) in an image framebeing processed. There may be various methods for calculating the degreeof reliability. For example, for each image frame included in a timewindow used for generating the background image described above, theratio at which each pixel or each set of pixels such as macroblock hasbeen determined to be a background by the foreground/backgroundseparation unit 120 may be calculated for each pixel or each set ofpixels such as macroblock and the ratio may be used as the degree ofreliability of the time window. The ratio at which a pixel or a set ofpixels has been determined to be background will be hereinafter alsoreferred to as the background-pixel acquisition ratio. Thebackground-pixel acquisition ratio is lowered in a region in which thebackground rapidly changes, such as a region in which a flow of movingobjects constantly occurs, and therefore it is difficult to properlydetect a stationary state from the background image. The reliabilitycalculation unit 140 may set a low degree of reliability for such aregion. The degree of reliability may be calculated for each of timewindows used for generating a plurality of background imagesrespectively.

The reliability calculation unit 140 may calculate the background-pixelacquisition ratio for each time window and then calculate the degree ofreliability by taking into consideration the background-pixelacquisition ratios. More specifically, for example, when thebackground-pixel acquisition ratios are low in all of the time windowsfor which the degree of reliability has been calculated, the region maybe a region from which background pixels are always unlikely to beacquired. The reliability calculation unit 140 may set a low degree ofreliability for such a region.

The reliability calculation unit 140 may calculate the degree ofreliability by using environmental changes. A case will be described inwhich the reliability calculation unit 140 receives an input ofenvironmental information such as day of week (for example a weekday,weekend, or the like), time, location, weather or the like which canchange a background. In this case, the reliability calculation unit 140may statistically calculate the degree of reliability depending onenvironmental conditions obtained from such environmental information,or by taking into consideration the background-pixel acquisition ratio.

Based on the result of stationary-state determination for each region(including a case of being each pixel) made by the stationary regionextraction unit 130 and on the degree of reliability calculated by thereliability calculation unit 140, the stationary-state likelihooddetermination unit 160 determines whether or not each region isstationary, or determines the likelihood of each region beingstationary. The stationary-state likelihood determination unit 160 maynot output the result of the stationary-state detection for a regionwith a degree of reliability lower than a threshold (may exclude theregion from the determination), for example. In this case, for a regionwith a degree of reliability higher than the threshold, thestationary-state likelihood determination unit 160 may output thedetection result of stationary region/non-stationary region, which isthe result of the stationary-state determination made by the stationaryregion extraction unit 130.

When the degree of reliability of a region is lower than the thresholdbut only the degree of reliability in a short-term time window is lowand the degrees of reliability in other time windows such as a long-termtime window are high, it is assumed that a moving object in a stationarystate is likely to be hidden behind another moving object for a shorttime, for example. The stationary-state likelihood determination unit160 may identify such a region as a region likely to be stationary.

A case will be described in which even when the degree of reliability ofa region subjected to the determination processing (a region beingprocessed) is low, a stationary state has been detected in a regionspatially close to the region being processed or in a correspondingregion in a close image frame (a temporally close region). In this case,the stationary-state likelihood determination unit 160 may identify theregion being processed is likely to be stationary.

The output unit 170 outputs, the results of determinations made by thestationary-state likelihood determination unit 160 that a stationarystate has occurred in a region, or that a stationary state has notoccurred in a region, or that a stationary state is likely to haveoccurred in a region, to a display device, a storage medium or anotherinformation processing device, for example.

(1.3 Process Flow)

A flow of process in the image processing system 100 will be describedbelow with reference to FIGS. 4 to 6.

Steps of the process described below can be executed in a differentorder or in parallel as appropriate, or another step may be addedbetween the steps unless inconsistency occurs in the processing details.Further, a step that is conveniently illustrated as a single step may beexecuted dividing the step into multiple steps, or steps convenientlyillustrated as being separate steps may be executed as a single step.

(1.3.1 Flow of Processing for Identifying Stationary Region by theStationary Region Extraction Unit 130)

A flow of processing for identifying a stationary region performed bythe stationary region extraction unit 130 will be described first withreference to FIG. 4.

The image input unit 110 receives an input of an image frame of videocaptured by a video camera or an image frame obtained by decoding videodata in which the video is recorded, for example (S401).

The foreground/background separation unit 120 separates the image frameinput from the image input unit 110 into a static, background region anda dynamic, foreground region (S403). The generated background regionimage is stored in the image storage unit 150, for example.

The background image generation unit 131 uses a background regionidentified by the foreground/background separation unit 120 in eachimage captured in a predetermined time window to generate a backgroundimage (S405). In this case, the background image generation unit 131generates background images in a plurality of time windows depending onthe stationary time of a moving object for which a stationary state isto be extracted.

The background image comparison unit 133 compares respective backgroundimages in the time windows generated by the background image generationunit 131 with one another (S407) and the stationary region determinationunit 135 identifies a region with a difference greater than a thresholdas a stationary region (S409).

(1.3.2 Flow of Processing for Calculation of Reliability)

A flow of processing for calculating the degree of reliability performedby the reliability calculation unit 140 will be described next withreference to FIG. 5. Note that a case in which the reliabilitycalculation unit 140 generates the degree of reliability based on thebackground-pixel acquisition ratio independently of environmentalconditions will be described here.

The reliability calculation unit 140 reads background regions in theimage frames included in the time windows for which the degrees ofreliability is to be calculated from the image storage unit 150 (S501).The background region in each image frame is identified by theforeground/background separation unit 120 as described above.

The reliability calculation unit 140 calculates, for each pixel, theratio of an acquired background region (background-pixel acquisitionratio) to each of the image frames included in the time window (S503).The reliability calculation unit 140 outputs the background-pixelacquisition ratio to the stationary-state likelihood determination unit160 as the degree of reliability (S505).

(1.3.3 Flow of Processing for Determining Stationary-State Likelihood)

A method for determining a stationary state by the stationary-statelikelihood determination unit 160 will be described with reference toFIG. 6.

The stationary-state likelihood determination unit 160 determineswhether or not each pixel or each set of pixels (hereinafter a pixel ora set of pixels will be collectively referred to as a region) is astationary region as illustrated in FIG. 6.

First, the stationary-state likelihood determination unit 160 determineswhether or not a region for which stationary-state determination is tobe made (a region being processed) has been determined to be astationary region by the stationary region extraction unit 130 (S601).If the region is determined to be a stationary region (YES at stepS601), the stationary-state likelihood determination unit 160 determineswhether or not the degree of reliability of the determination to be astationary region calculated by the reliability calculation unit 140, ishigher than a threshold (S603). If the degree of reliability is higherthan the threshold (YES at S603), the stationary-state likelihooddetermination unit 160 determines that the region is stationary (S605).

If the degree of reliability of the region being processed is less thanor equal to the threshold (NO at S603), the stationary-state likelihooddetermination unit 160 determines that the degree of reliability of theresult of the determination is low even though the region is determinedto be stationary by the stationary region extraction unit 130, andexcludes the region from the stationary-state determination (S607).

If it is determined at step S601 that the region being processed is nota stationary region at step S601 (NO at step S601), the stationary-statelikelihood determination unit 160 performs processing similar to theprocessing at S603. Specifically, the stationary-state likelihooddetermination unit 106 determines whether or not the degree ofreliability of the determination that the region is stationarycalculated by the reliability calculation unit 140 is higher than athreshold (S609). If the degree of reliability is higher than thethreshold (YES at step S609), the stationary-state likelihooddetermination unit 160 determines that the region is not stationary(S611).

On the other hand, if the degree of reliability is less than or equal tothe threshold (NO at S609), the stationary-state likelihooddetermination unit 160 determines whether or not a region spatially ortemporally close to the region being processed has been determined to bestationary (S613). The temporally close region is a region in a locationcorresponding to that of the region being processed in a temporallyclose image frame. If a temporality or spatially close region has beendetermined to be stationary (YES at S613), the stationary-statelikelihood determination unit 160 determines that the region is likelyto be stationary and may output the region in distinction fromstationary and non-stationary regions (S615). If a temporally orspatially close region has not been determined to be stationary (NO atS613), the stationary likelihood determination unit 160 excludes theregion from the determination (S617).

(1.4 Hardware Configuration)

An example of hardware configuration for implementing the imageprocessing system 100 described above by a computer will be describedbelow with reference to FIG. 7. As noted previously, the functions ofthe image processing system 100 may be implemented by a plurality ofinformation processing devices.

As illustrated in FIG. 7, the image processing system 100 includes aprocessor 701, a memory 703, a storage device 705, an input interface(I/F) 707, a data I/F 709, a communication I/F 711, and a display device713.

The processor 701 controls various processes in the image processingsystem 100 by executing programs stored in the memory 703. For example,processes relating to the image input unit 110, theforeground/background separation unit 120, the stationary regionextraction unit 130, the reliability calculation unit 140, thestationary-state likelihood determination unit 160 and the output unit170 can be implemented as a program temporarily stored in the memory 703and then operated primarily on the processor 701.

The memory 703 is a storage medium such as a random access memory (RAM).The memory 703 temporarily stores program code of a program executed bythe processor 701 and data required during execution of the program. Forexample, a stack area required during execution of the program isreserved in the memory area in the memory 703.

The storage device 705 is a nonvolatile storage medium such as a harddisk or a flash memory, for example. The storage device 705 stores anoperating system, various kinds of programs for implementing the imageinput unit 110, the foreground/background separation unit 120, thestationary region extraction unit 130, the reliability calculation unit140, the stationary-state likelihood determination unit 160, and theoutput unit 170, and various kinds of data including the image storageunit 150. The programs and data stored in the storage device 705 areloaded into the memory 703 as necessary and accessed by the processor701.

The input I/F 707 is a device for accepting inputs from a user. Examplesof the input I/F 707 include a keyboard, a mouse, a touch panel and thelike. The input I/F 707 may be connected to the image processing system100 via an interface such as a universal serial bus (USB), for example.

The data I/F 709 is a device for inputting data from outside the imageprocessing system 100. Examples of the data I/F 709 include drivedevices for reading data stored on various storage media and the like.The data I/F 709 may be provided outside the image processing system100. In that case, the data I/F 709 is connected to the image processingsystem 100 via an interface such as a USB, for example.

The communication I/F 711 is a device for providing data communicationwith devices external to the image processing system 100, for example, avideo camera and the like, via wire or wirelessly. The communication I/F711 may be provided outside the image processing system 100. In thatcase, the communication I/F 711 is connected to the image processingsystem 100 via an interface such as a USB, for example.

The display device 713 is a device for displaying various kinds ofinformation. Examples of the display device 713 include a liquid-crystaldisplay, an electro-luminescence (EL) display and the like, for example.

The display device 713 may be provided outside the image processingsystem 100. In that case, the display device 713 is connected to theimage processing system 100 via a display cable or the like, forexample.

(1.5 Advantageous Effects of the Present Example Embodiment)

As described above, the image processing system 100 according to thepresent example embodiment not only makes stationary-state determinationbut also calculates the degree of reliability of the stationary-statedetermination. By using the degree of reliability for thestationary-state determination, false detection of a stationary statecan be reduced and lost or discontinuous stationary regions can becomplemented, for example. Thus, the accuracy of the stationary-statedetection can be improved.

Further, the result of the stationary-state detection can be used todetect suspicious persons (loiterers, stationing persons, ambushingpersons, persons sleeping on the street, and the like) and suspiciousobjects for the purpose of security. In addition, stationary-stateinformation about customers at each of the shelves (staying time,frequency, and the like) can be collected for the purpose of marketing.

(2 Second Example Embodiment)

A second example embodiment will be described below with reference toFIG. 8. FIG. 8 is a block diagram illustrating a functionalconfiguration of an image processing system 800. As illustrated in FIG.8, the image processing system 800 includes a determination unit 810 anda generation unit 820.

Based on a first image generated from each of image frames captured in afirst time period from the time of processing and a second imagegenerated from each of image frames captured in a second time periodfrom the time of the processing, the determination unit 810 determineswhether or not a region is a stationary region in an image frame at thetime of processing. Here, the first time period is a short-term timewindow and the second time period is a long-term time window describedin FIG. 2, for example. The first time period and the second time periodare different from each other. The first image and the second image arebackground images in the respective time periods, for example.

The generation unit 820 generates reliability information aboutdetermination as to whether or not each region in an image frame beingprocessed is a stationary region.

The image processing system 800 according to the example embodimentimplemented as described above enables proper detection of a stationarystate.

(3 Additional Matters)

Some of the components of the configurations of the example embodimentsdescribed above may be combined or replaced. Further, the configurationof the present invention is not limited to the example embodimentsdescribed above; various modifications can be made without departingfrom the spirit of the present invention.

Some or all of the example embodiments described above can be alsodescribed as, but not limited to, the following supplementary notes. Aprogram of the present invention may be a program for causing a computerto execute the respective operations described in the exampleembodiments.

(Supplementary Note 1)

An image processing system including: determining means for determiningwhether or not a region is a stationary region in an image frame at thetime of processing, based on a first image generated from each imageframe captured in a first time period from the time of the processingand a second image generated from each image frame captured in a secondtime period from the time of the processing, the second time periodbeing longer than the first time period; and generation means forgenerating reliability information about determination as to whether ornot each region in an image frame being processed is the stationaryregion.

(Supplementary Note 2)

The image processing system according to Supplementary Note 1, whereinthe generation means generates, for each region in the image frame beingprocessed, the reliability information based on a ratio at which acorresponding region in each image frame captured in the first timeperiod is a static region.

(Supplementary Note 3)

The image processing system according to Supplementary Note 1 or 2,wherein the generation means generates the reliability information inaccordance with an environmental condition.

(Supplementary Note 4)

The image processing system according to any one of Supplementary Notes1 to 3, further including means for, based on the reliabilityinformation, including or excluding each region in the image frame beingprocessed into or from the determination as to whether or not the regionis the stationary region.

(Supplementary Note 5)

The image processing system according to Supplementary Note 4, furtherincluding means for, when the reliability information of a region beingprocessed is lower than a threshold and a region spatially close to theregion being processed or a corresponding region in an image frametemporally close to an image frame at the time of the processing is thestationary region, determining the region being processed to be a regionlikely to be stationary.

(Supplementary Note 6) An image processing method including: a step ofdetermining whether or not a region is a stationary region in an imageframe at the time of processing, based on a first image generated fromeach image frame captured in a first time period from the time of theprocessing and a second image generated from each image frame capturedin a second time period from the time of the processing, the second timeperiod being longer than the first time period; and a step of generatingreliability information about determination as to whether or not eachregion in an image being processed is the stationary region.

(Supplementary Note 7)

The image processing method according to Supplementary Note 7, whereinfor each region in the image frame being processed, the reliabilityinformation is generated based on a ratio at which a correspondingregion in each image frame captured in the first time period is a staticregion.

(Supplementary Note 8)

The image processing method according to Supplementary Note 6 or 7,wherein the reliability information is generated in accordance with anenvironmental condition.

(Supplementary Note 9)

The image processing method according to any one of Supplementary Notes6 to 8, further including means for, based on the reliabilityinformation, including or excluding each region in the image frame beingprocessed into or from the determination as to whether or not the regionis the stationary region.

(Supplementary Note 10)

The image processing method according to Supplementary Note 9, furtherincluding the step of, when the reliability information of a regionbeing processed is lower than a threshold and a region spatially closeto the region being processed or a corresponding region in an imageframe temporally close to an image frame at the time of processing isthe stationary region, determining the region being processed to be aregion likely to be stationary.

(Supplementary Note 11)

A program which causes a computer to execute the processes of:determining whether or not a region is a stationary region in an imageframe at the time of processing, based on a first image generated fromeach image frame captured in a first time period from the time of theprocessing and a second image generated from each image frame capturedin a second time period from the time of the processing, the second timeperiod being longer than the first time period; and generatingreliability information about determination as to whether or not eachregion in an image frame being processed is the stationary region.

(Supplementary Note 12)

The program according to Supplementary Note 11, wherein for each regionin the image being processed, the reliability information is generatedbased on a ratio at which a corresponding region in each image framecaptured in the first time period is a static region.

(Supplementary Note 13)

The program according to Supplementary Note 11 or 12, wherein thereliability information is generated in accordance with an environmentalcondition.

(Supplementary Note 14)

The program according to any one of Supplementary Notes 11 to 13,further including means for, based on the reliability information,including or excluding each region in the image frame being processedinto or from the determination as to whether or not the region is thestationary region.

(Supplementary Note 15)

The program according to Supplementary Note 14, causing the computer tofurther execute the process of, when the reliability information of aregion being processed is lower than a threshold and a region spatiallyclose to the region being processed or a corresponding region in animage frame temporally close to an image frame at the time of processingis the stationary region, determining the region being processed to be aregion likely to be stationary.

This application is based upon and claims the benefit of priority fromJapanese Patent Application No. 2014-159045, filed on Aug. 4, 2014, theentire disclosure of which is incorporated herein.

REFERENCE SIGNS LIST

100 Image processing system

110 Image input unit

120 Foreground/background separation unit

130 Stationary region extraction unit

131 Background image generation unit

133 Background image comparison unit

135 Stationary region determination unit

140 Reliability calculation unit

150 Image storage unit

160 Stationary-state likelihood determination unit

170 Output unit

701 Processor

703 Memory

705 Storage device

707 Input interface

709 Data interface

711 Communication interface

713 Display device

800 Image processing system

810 Determination unit

820 Generation unit

1-7. (canceled)
 8. A stationary-state detection system comprising: atleast one memory storing instructions; and at least one processorcoupled to the at least one memory and configured to execute theinstructions to: generate a first image based on a plurality of imageframes captured in a first time period starting from a time ofprocessing and generate a second image based on a plurality of imageframes captured in a second time period starting from the time ofprocessing, the second time period being longer than the first timeperiod; extract one or more stationary regions from an image frame atthe time of processing, based on the first image and the second image, astationary region being a region in which a moving object is stationary;generate reliability information as to the extraction for each region inthe image frame at the time of processing; perform determination ofwhether a target region in the image frame at the time of processing isa stationary region, non-stationary region or a region likely to be astationary region, based on the reliability information; and output theregion determined to be a region likely to be a stationary region indistinction from both the region determined to be a stationary regionand the region determined to be a non-stationary region.
 9. Thestationary-state detection system according to claim 8, wherein theprocessor is configured to execute the instructions to determine thatthe target region is a region likely to be a stationary region when thereliability information for the target region does not meet thecriterion and a predetermined condition is satisfied, the predeterminedcondition comprising a condition that a region spatially close to thetarget region or a corresponding region in an image frame temporallyclose to the image frame at the time of processing is determined to be astationary region.
 10. The stationary-state detection system accordingto claim 8, wherein the processor is configured to execute theinstructions to determine that the target region is a region likely tobe a stationary region when the reliability information for the targetregion does not meet the criterion and a predetermined condition issatisfied, the predetermined condition comprising a condition thattarget region is not extracted as a stationary region.
 11. Thestationary-state detection system according to claim 8, wherein theprocessor is further configured to generate the reliability informationfor the target region based on a ratio of stationary regions to regionscorresponding to the target region in image frames captured in the firsttime period.
 12. A stationary-state detection method comprising:generating a first image based on a plurality of image frames capturedin a first time period starting from a time of processing and generate asecond image based on a plurality of image frames captured in a secondtime period starting from the time of processing, the second time periodbeing longer than the first time period; extracting one or morestationary regions from an image frame at the time of processing, basedon the first image and the second image, a stationary region being aregion in which a moving object is stationary; generating reliabilityinformation as to the extraction for each region in the image frame atthe time of processing; performing determination of whether a targetregion in the image frame at the time of processing is a stationaryregion, non-stationary region or a region likely to be a stationaryregion, based on the reliability information; and outputting the regiondetermined to be a region likely to be a stationary region indistinction from both the region determined to be a stationary regionand the region determined to be a non-stationary region.
 13. Thestationary-state detection method according to claim 12, wherein thedetermination comprising determination of that the target region is aregion likely to be a stationary region when the reliability informationfor the target region does not meet the criterion and a predeterminedcondition is satisfied, the predetermined condition comprising that aregion spatially close to the target region or a corresponding region inan image frame temporally close to the image frame at the time ofprocessing is determined to be a stationary region.
 14. Thestationary-state detection method according to claim 12, wherein thedetermination comprising determination of that the target region is aregion likely to be a stationary region when the reliability informationfor the target region does not meet the criterion and a predeterminedcondition is satisfied, the predetermined condition comprising acondition that target region is not extracted as a stationary region.15. The stationary-state detection method according to claim 12,comprising generating the reliability information for the target regionbased on a ratio of stationary regions to regions corresponding to thetarget region in image frames captured in the first time period.
 16. Anon-transitory computer-readable storage medium storing a program thatcauses a computer to execute: generating a first image based on aplurality of image frames captured in a first time period starting froma time of processing and generate a second image based on a plurality ofimage frames captured in a second time period starting from the time ofprocessing, the second time period being longer than the first timeperiod; extracting one or more stationary regions from an image frame atthe time of processing, based on the first image and the second image, astationary region being a region in which a moving object is stationary;generating reliability information as to the extraction for each regionin the image frame at the time of processing; performing determinationof whether a target region in the image frame at the time of processingis a stationary region, non-stationary region or a region likely to be astationary region, based on the reliability information; and outputtingthe region determined to be a region likely to be a stationary region indistinction from both the region determined to be a stationary regionand the region determined to be a non-stationary region.
 17. The storagemedium according to claim 16, wherein the determination comprisingdetermination of that the target region is a region likely to be astationary region when the reliability information for the target regiondoes not meet the criterion and a predetermined condition is satisfied,the predetermined condition comprising that a region spatially close tothe target region or a corresponding region in an image frame temporallyclose to the image frame at the time of processing is determined to be astationary region.
 18. The storage medium according to claim 16, whereinthe determination comprising determination of that the target region isa region likely to be a stationary region when the reliabilityinformation for the target region does not meet the criterion and apredetermined condition is satisfied, the predetermined conditioncomprising a condition that target region is not extracted as astationary region.
 19. The storage medium according to claim 16, whereinthe program further causes the computer to perform generating thereliability information for the target region based on a ratio ofstationary regions to regions corresponding to the target region inimage frames captured in the first time period.